A digital camera is often required to estimate or detect the illumination condition while a user is taking a picture. Then the digital camera needs to perform automatic white balancing in order to compensate for the color deviation caused by the extreme illumination. For example, a white paper under yellow light will be image captured as yellow, even though the desired result is to show the white paper as white. As such, the digital camera should perform white balancing in order to make the white paper look “white” even under the yellow light.
Several auto white balancing techniques have been proposed in the past. Examples include the widely used gray-world algorithm, the RGB max algorithm, neural network approaches, and “color in perspective” approaches.
Among these auto white balancing techniques, “color in perspective” technique is reported to have superior performance. An image sensor's responses are recorded for various color temperatures (light sources), with each color temperature tested with a plurality of different surfaces.
Specifically, as each color temperature is shone on a variety of surfaces, the resulting pixel values sensed by the image sensor can be plotted in a 3-dimensional color space (e.g., a RGB space). As the sensor's response to a particular color temperature, these pixel values can be seen as forming a “gamut” or a cluster in this color space. With a different color temperature, the sensor's response to this color temperature is represented as another gamut (i.e., a cluster) formed in this color space. In turn, the sensor's responses to respectively n color temperatures are represented respectively as n different gamuts in the color space. These resulting “reference” gamuts then serve as references with which to perform white balancing on sensed images.
In theory, as the image sensor captures an image, the pixel values of the captured image also form a gamut for that image in the color space. Out of the n reference gamuts, the reference gamut having the highest correlation with the image's gamut is selected. The color temperature associated with this selected reference gamut is used as the source light for white balancing the captured image.
In practice, before the gamut selection is to take place, buffer limitation necessitates 2-dimensional representations of the n reference gamuts. Also, buffer limitation necessitates a 2-dimensional representation of the gamut that corresponds to the pixel values from the captured image. Information is inevitably lost from this “3D to 2D” projection. Therefore, to select a right direction of the projection is crucial to performing white balancing.
The conventional techniques project the 3D gamuts in the color space onto a 2D “chroma” surface. In other words, luminance information is discarded. However, due to the nature of digital imaging sensors and the characteristics of optical filters, the luminance dimension actually contains considerable information for distinguishing different illuminations (light sources).
For example, one research group discovered that if the gamuts are projected onto the sensor's raw red and blue surface (RB plane) rather than pure chroma surface (ab plane in Lab color space, or uv plane Yuv color space), the color separability is actually improved. However, the RB plane might not be the near-optimal 2D surface onto which to project the reference gamuts. As such, in the present invention, a system and method are proposed to systematically select the near-optimal 2D surface onto which to project the reference gamuts.